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ABSTRACT 


In this paper, we describe the construction of important landmarks of roads in 
the GIS environment. The system uses the corners between more than two 
roads as an important landmarks. In this corner points will be saving a 
number of images, each one represents the movement direction between two 
segment roads. The objective of our work is to build the geo-database 


repository depend on the GIS (vector data) and multimedia (raster data) 
information. This paper considered as a preprocessing step for a roadmap 
Keywords: discovery of video film when the input to the system is a dolly video film of 
roads with absence the GPS locations of frames video, and the output is the 


Geo-localization pathway (route) on the map inside the GIS environment. This work 


GIS considered as a starting point for multimedia query inside the GIS 
Images environment. That’s mean when query the GIS system about a particular 
Maps image or video, the GIS system must be able to know and determine the 
Video location of this image or video file on the map. Thus, the first step for doing 
this process needs to building and constructing an appropriate database for 

matching process later. 
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1, INTRODUCTION 

GIS (Geographic Information System) are generally well-defined as an information system that 
analyzes, manipulates and manages the spatial data. GIS is used to produce information that is useful in 
decision making [1-6]. A GIS depends on an organized database that describes the world in geographic 
positions. A GIS is a unique kind of database of the world that represent a geographic database 
(geo-database), As part of a GIS geo-database design, users specify how certain features will be represented 
of ordered collections of vector-based features, Raster datasets, networks, terrains, and other surfaces and 
survey data sets. GIS datasets include traditional tabular attributes that define the geographic objects. 
These tabular information sets and relationships play a key role in GIS data models, just as they do in 
traditional database applications [7, 8]. 

The central to the GIS system is the database that is a collection of maps and associated information 
in digital form. Since the database is concerned with earth surface features, 1t can be seen to be involved of 
two components that are a spatial database describing the geography (shape and position) of earth surface 
features, and an attribute database describing the characteristics or qualities of these features. 
Thus, for example, we might have a property parcel defined in the spatial database and qualities such as its 
land use, owner, property valuation, and so on, in the attribute database [9, 10]. 

GIS fundamental component representations of spatial data are frequently well-defined in terms of 
points, lines, and areas. Nevertheless, additional kind of spatial data that 1s becoming commonly captured, 
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but as yet is largely ignored in GIS, 1s that of video and images. While the digital video recording is a usually 
encountered medium in modern society and encompasses many forms, from the simple personal camera 
through to sophisticated survey and surveillance systems, its geographical representation in a GIS has not 
been fully examined or realized. In the majority of cases, the video film is commonly captured while the 
device and/or the objects being observed are in motion. What is of certain interest is when video streams can 
be, or have been, associated with spatial data such as location and orientation to generate geographically 
referenced video graphic data, which, for simplicity, will be defined as spatial video. Fundamentally, 
the nature of video is to record space, so when spatial properties can be accurately acquired and associated 
with this footage, an important geographical element can be considered for integration and analysis within a 
GIS [11, 12]. 

The media that represent spatial data may be extended to various media types such as aerial photo, 
graphics, attribute image, and satellite imagery. Such data types make users have richer information with 
various and complementary characteristics [13, 14]. Modeling video in GIS has been largely ignored for 
numerous reasons such as data volumes involved in collecting and storing video footage. Thus, this paper 
suggests to indexing and storing small important video frames not exceeded 1% of the number of video 
frames in the GIS environment. 

Recently, the most camcorders are mounted with GPS and enabled the linkage between video and 
GPS. Most videos present only graphics without a position of such video [15, 16]. If the video can be 
visualized with its traveling path on GIS and snapshot of video can be gotten with its location, this will be 
very useful for GIS people to make use this media for updating GIS data [17, 18]. 

The basic motivation of this paper is how to build a database that can be used for determining the 
spatial location (geographic information) of a video or images imagine by a camcorder which is not mounted 
with GPS device in the GIS environment. 

In the organization of the paper, Section 2, the proposed framework, which describes how the 
methodology presented Section 3, Experimental Results, gives an explanation of the output route 
highlighting and images displayed for important landmarks. In Section 4, Conclusions, where the authors 
reflect on the used method, achieved results and further improvements. 


2. THE PROPOSED FRAMEWORK 
The block diagram of the proposed framework for this paper is illustrated in Figure | that consists of 
many steps. 
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Figure 1. The block diagram of the proposed framework 


2.1. Map of Interested Area 

In the first step, we determine the workspace of the interested area inside the GIS environment. 
The information on the map (the interested area) are exported from the OpenStreetMap platform [19-21]. 
This platform can be given freely the information about (points that represent the important landmarks such 
as schools, hospital and government facilities, line that represent the roads, polygon that represents building) 
for any interested area in the world. The class features that used in our system is the points and lines. 
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The data on the roadmap are extracted as a shape file of a polyline. From this shape file extracted their 
feature vertices and converted to its point features shape file. The feature points may need some rectification 
process by removing or adding some points feature. The output of this step is two shape files, the first one is 
a point feature that represent an intersection between at least two segment road (line), while the second one is 
the line features that represent the line between two intersection points. 


2.2. Images of Texture for Each Intersection 

In this step, will be taken a picture for each important landmark as a forward facing camera (the 
dolly forward movement [22-25]). The important landmark represents the intersection of at least two roads. 
The intersection can be represented as a point in the GIS environment. If the intersection point between two 
roads, that means we need to take two pictures for each side such as illustrated in Figure 2. 











Figure 2. Taking a picture shot for an intersection between two roads 


The picture is taken when there is a rotation to the left or to the right direction. We note in 
Figure 2-A the picture is taken for the texture environment at the moment that a car 1s moved from forward 
moving and change its direction to the right direction. In the Figure 2-B the picture is taken for the texture 
environment at the moment that a car is moved from forward moving and change its direction to the left 
direction. The common point (intersection or important landmark) in the above scenario has two pictures or 
images if the road has a two-way street, and one image if the road has one-way street. Now, what about if the 
important landmark represents the intersection between four roads. In this situation, the system needs to take 
(8) pictures for each direction change. Figure 3 explains this scenario. 





Figure 3. Taking pictures shot for an intersection between four roads 


We notice in the Figure 3-A the car is moving from bottom to up. There are three possible moving 
of car before reach the intersection point, the first one is forward moving direction that does not need to take 
a picture shot, the second one is moving to the right direction which needs to take a picture shot, the last 
possible moving is to the left direction which also needs to take a picture shot. 
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2.3. Features Extraction and Construction of Dictionary 

In this step will be extracted the important landmarks on the roadmap in GIS and build the database 
table. There are two types of feature class are used for roads in the GIS environment. The first one is the 
point feature class that represents the intersection between more than two roads. The second one is the line 
feature class that represent road or segment between two intersection points. In our system, the road or street 
can be divided into the number of segments that represent the line between two intersections such as 
Figure 4. 





Figure 4. Example of roadmap representation 


We notice from the Figure 4, there are six main roads (road 1, road 2, ..., road 6). Each road has a 
number of intersection points that represent the important landmarks and also a number of segments. 
From the figure above, each of road (1) and road (2) has four intersection points and four segments. All of the 
road (3), road (4) and road (5) has two important landmarks and segments. The road (6) has two intersection 
points and one segment. The road (1) has four important landmarks that represent the intersection points 
(5, 6, 7, 8), and four segments (7, 9, 11, 14). The other roads can be interpreted in the same way. 
Each segment represents the line or road between two intersection points. Each intersection point represents 
the cross between more than two segments. If the intersection point between only two segments that means 
we need to save two images associated with this point feature class such as points (4 and 8) in the Figure 4. 
Each image represents one segment of road, in the case of point (4) the first image represents the segment 
(13) and the second image represent the segment (16). If the intersection point between three segments that 
means we need to save four images associated with this point feature class such as points (1, 2 and 3). In the 
case of point (2), the first image represents segment (3), that means the picture is taken when moving from 
segment (4) to segment (3). The second image represents the segment (5), that means the picture is taken 
when moving from segment (4) to segment (5). The third and fourth images represent the segment (4), that 
means the third picture is taken when moving from segment (3) to segment (4) and the fourth picture 1s taken 
when moving from segment (5) to segment (4). If the intersection point between four segments that means 
we heed to save eight images associated with this point feature class such as points (5, 6 and 7). In the case of 
point (5), each segment represented by two images such as Figure 3. Now we know how many numbers of 
images are needed to saving for each intersection points in the roadmap. The final operation of this step is to 
create a dictionary of important landmarks(intersection points) and each one has a number of segment 
connected to this intersection point and number of images that represent the neighbor segments. 
Table | represent the final output of this step as dictionary table (the database). 


Table 1. The Dictionary Constriction of Figure (4) 


Important landmarks Segment connected No. of images Images 
Point | 1,2,3 4 1,2,3,4 
Point 2 3,4,5 4 5,6,7,8 
Point 3 5,6,13 4 9,10,11,12 
Point 4 13,16 2 13,14 
Point 5 7,8,9,12 8 15,16,17,18,19,20,21,22 
Point 6 4,9,10,11 8 23,24,25,26,27,28,29,30 
Point 7 6,11,12,14 8 31,32,33,34,35,36,37,38 
Point 8 14,16 2, 39,40 
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2.4. The Input Route 

From the above steps, we extracted the important landmarks features from the roadmap of the 
interested area and taking a number of real pictures shot to these landmarks to build the database. The input 
to the system is a pathway depends on the segment number of the road rout. In Figure 5, there are three cases 
of the input rout step when the route (1) has the segments number (1, 2, 9, 4, 5), while the route (2) has the 
segments number (16, 14, 12) and route (3) has the segments number (8, 9, 4, 3, 2, 7) consecutively. 


B 
A Routel Route 2 Route 3 
1,2,9,4,5 16,14,12 8,9,4,3,2,7 


Segment 12 


Segment 14 


Segment 2 
Segment 16 





Figure 5. Example of input routes 


2.5. Displaying the Image Features of a Pathway 

In this step matching the segment road number of the above step with the constructed database for 
highlight the pathway and display the corresponding images when each image represents the movement from 
one segment to another. For example in the case (A) of the Figure 5, the input route is (1, 2, 9, 4, 5). 
The first two segments are taken and extract the corresponding intersection point between them in the 
dictionary. We note the point (1) is the intersection between the sequential segment and it has four saving 
images (1-4) from the Table 1. The image (1) represents the view between the segment (1 2), while the 
image (2) represents the view shot between the segment (3 2), images (3, 4) represent the view between the 
segments (2 > 1) and (2 > 3) consecutively. Thus, the image (1) is displaced. The next two segments are 
(2, 9), thus the point between them segment 1s a point (5). We notice from the Table | there are eight images 
(15-22) saving in the dictionary each one represents the view or moving direction between two segments. 
These images have represented in the Table 2. 


Table 2. The Images Representation of the Intersection Point (5) 








Images From segment To segment 
15 Segment 2 Segment 7 
16 Segment 2 Segment 9 
17 Segment 7 Segment 2 
18 Segment 7 Segment 8 
19 Segment 8 Segment 7 
20 Segment 8 Segment 9 
21 Segment 9 Segment 2 
22 Segment 9 Segment 8 





From Table 2 we notice the image between (segment 2 > segment 9) is an image (16). Next, take 
the next two consecutive segments (9, 4) to find the common point between them that is a point (6). 
We notice from the Table | there are eight images (23-30) saving in this intersection point. These images are 
represented in a Table 3. 
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Table 3. The Images Representation of Intersection Point (6) 








Images From segment To segment 
23 Segment 4 Segment 9 
24 Segment 4 Segment 11 
2) Segment 9 Segment 4 
26 Segment 9 Segment 10 
27 Segment 10 Segment 9 
28 Segment 10 Segment 11 
29 Segment 11 Segment 4 
30 Segment 11 Segment 10 
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From Table 3 we notice the image between (segment 9 > segment 4) is the image (25). Next, take 
the next two consecutive segments (4, 5) to find the common point between them that is a point (2). 
We notice from the Table 1 there are four images (5-8) saving in this intersection point. These images have 
represented in the Table 4. 


Table 4. The Images Representation of the Intersection Point (2) 








Images From segment To segment 
5 Segment 3 Segment 4 
6 Segment 4 Segment 3 
7 Segment 4 Segment 5 
8 Segment 5 Segment 4 





From Table 4 we notice the image between (segment 4 > segment 5) is the image (7). Now, the 
output of the proposed framework is to highlight or flashing the segments input on the map and displaying 
list if images on that pathway or route. The list of images for the route of the case (A) on the Figure 5 that 
displayed on the map is (image I, image 16, image 25 and image 7) such as illustrated in the Figure 6. 





Figure 6. The output display of the route (1, 2, 9, 4, 5) 


For map display output of other cases routes in Figure 5 can be extracted as in the manner described 
above. Figure 7 and 8 illustrate the output of a map with images for the route (16, 14, 12) and route 
(8, 9, 4, 3, 2, 7). 





Figure 7. The output display of the route (16, 14, 12) 
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Road 5 Road 4 Road 3 





Figure 8. The output display of the route (8, 9, 4, 3, 2, 7) 


3. EXPERIENTIAL RESULTS 

Experiments of the proposed framework in this paper are performed on a small region of Karlsruhe 
city Baden-Wiirttemberg-Germany. The information about roadmap is extracted from the OpenStreetMap 
platform as explains in Figure 9. 





Figure 9. The interested area of Karlsruhe city from the OSM map. 


There are many important landmarks (in our framework the important landmarks represent the 
intersection between roads). The intersection points are determined manually and take a number of picture 
shot for each point depends on the number of segment road connected to that point. Figure 10 represent the 
locations of the important landmarks on the map. 








Figure 10. The important landmarks of the interested area. 
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Pictures ware saved in these intersection points to build the dictionary database where each image 
has the location (X,Y) of their point affiliated to it. Each point may take (2, 4 or 8) images according to the 
segment road connected. There are (102) important landmarks point, (138) segment roads and (228) picture 
are saved for intersection points for the workspace area. Table 5 explains the dictionary of important 
landmarks. 


Table 5. The Dictionary Constriction of the Interested Area in the Figure (10) 








Point Segment road number Images 
1 1,2,3,4 1,2,3,4,5,6,7,8 
2 2,5,6 9,10,11,12 
5 3,1 13,14 
4 4,5,8,9 15,16,17,18,19,20,21,22 
102 136,137,138 225,226,227 ,228 





Now, to test our framework when the input route is (71, 69, 67, 64, 62, 59, 58, 56) segments. 
Figure 11 explain the final output that represents the highlighting the pathway route and important landmarks 
and also displays the images if you click on the intersection point. 
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Figure 11. The output of the roadmap with important landmarks images 


4. CONCLUSIONS AND FUTURE WORKS 

An efficient fusion framework was effectively built and proved by combining two types of data 
environments, the first one is the GIS environment such as (maps, features class and etc.), while the second 
one is the multimedia environment such as (images and videos). The goal of this paper is how to build a 
database that can be used for determining the geographic information of a video. This work considered the 
starting point for multimedia query inside the GIS environment. That’s mean when querying the GIS system 
about a particular image, the GIS system must know the location of this image on the map. In the future 
works, the construction database of this work can be used for extracting the locations on the GIS-map for 
road segments of the video film. The images that stored on the map dictionary may be used for the matching 
process with frames of video film based on appointed techniques. 
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